A New Approach to Assessing Test Bias

A new regression-based method of assessing test bias is proposed. We highlight two different potential causes of differences in groups' regression line intercepts. Intercepts differing due to mean criterion score differences are not interpreted as predictive test bias. Using both simulated and employee data, we illustrate this new approach. Test bias is a systematic error in how a test measures members of a particular group (Camilli & Shepard, 1994). Test bias is a fundamentally important issue in testing as pervasive and systematic sources of error can lead to erroneous inferences regarding the interpretation and use of test scores. Test bias may preclude comparisons across groups and is cause for considerable concern in decision making contexts such as employee selection (U.S. EEOC, 1978). The most common method of assessing test bias is the Cleary (1968) method, which involves testing for both slope and intercept regression line differences across groups (Sackett & Wilk, 1994). In this approach, the test serves as the predictor variable and some measure of performance serves as the criterion. The Cleary approach is widely used and endorsed by both the Standards for Educational and We briefly review the Cleary (1968) approach for assessing test bias, delineate two fundamentally different causes of group intercept differences, discuss potential reasons for these differences, and recommend a new approach to assessing test bias that allows for the distinction of intercept differences that can be said to be due to test bias and those that may be due to other causes. Significantly, we propose a method by which differences in group intercepts may not be interpreted as test bias. approach to assessing test bias involves regressing a criterion such as performance on a predictor (i.e., the test), a dichotomous group membership variable, and the cross-product of the predictor and group membership in a single regression model (Sackett, Laczo, & Lippe, 2003). Using this framework, if the group membership variable is significant, the groups are said to have regression lines with different intercepts. If the group-predictor cross-product is significant, groups have regression lines with different slopes. We only examine differences in groups' regression line intercepts. Like others (e.g., Sackett & Wilk, 1994), we agree that differential slopes indicate differential test validity which is problematic for any test. While the Cleary (1968) approach to assessing test bias has been widely used with ample success (Schmidt, 1988; Wigdor & Garner, 1982), we contend that there are …